Using a Musical Instrument as a Video Game Controller

ABSTRACT

The use of musical instruments as methods for controlling video games is disclosed. By translating the notes, beats, chords, note changes and other musical elements into input that can be recognized by a video game or video game system, the person playing the instrument will have the ability to play a game through the use of an instrument. This includes the ability to configure the music translation system to provide different input to the video game or video game system depending on the key in which the person is playing the instrument.

The use of a musical instrument to control action in a video game bytranslating musical information to an input format that can beunderstood by a video game system.

BACKGROUND OF THE INVENTION

There are currently video games in existence that provide users with thesimulated experience of playing musical instruments. The Guitar Hero®series and the game Rock Band® are particular examples of this genre.However, these games only provide the simulated experience of playing aninstrument.

For people who can play instruments or people who wish to learninstruments, these games only provide the most rudimentary idea of chordprogressions and notes. Thus, if a guitar player wishes to play one ofthese music games, the guitar player must learn an entirely foreigncontrol system. This is also the case for other musicians who play musicsimulations, such as bass players and drummers/percussionists.

A person who can play the instrument being simulated may wish toactually play the instrument itself, rather than attempt to learn how tooperate a controller that does not provide an experience like the actualinstrument. Likewise, if a person playing the game wishes to learn theactual instrument without losing the entertainment value of playing avideo game simulation that the person enjoys, there is not currently away for the person to translate the simulation to a real-worldexperience with real instruments.

BRIEF SUMMARY OF THE INVENTION

This invention is the use of using a musical instrument as an inputdevice for controlling a video game. By translating the notes or beatsbeing created by the musical instrument into input informationrecognizable to a video game or video game system, a musical instrumentcan be used to control the action or events in a video game.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a guitar connected by cable to a device thattranslates musical notes to video game input that is then sent through acable connection to a video game system.

FIG. 2A is a diagram of a guitar connected via wireless to a device thattranslates musical notes to video game input that is then sent through acable connection to a video game system.

FIG. 2B is a diagram of a guitar connected via wireless to a device thattranslates musical notes to video game input that is then sent via awireless connection to a video game system.

FIG. 2C is a diagram of a guitar connected by cable to a device thattranslates musical notes to video game input that is then sent via awireless connection to a video game system.

FIG. 3A is a diagram of a guitar pickup connected to a device thattranslates musical notes to video game input that is then sent to avideo game system.

FIG. 3B is a diagram of a guitar playing into a microphone that isconnected to a device that translates musical notes to video game inputthat is then sent to a video game system.

FIG. 4A is a diagram of a snare drum with a sensor attached and anelectronic drum connected to a device that translates drumbeats intovideo game input that is then sent to a video game system.

FIG. 4B is a diagram of an entire drum set with sensors attached toevery component. Every beat picked up by the sensors is sent to a devicethat translates the drumbeats into video game input that is then sent toa video game system.

FIG. 5A is a diagram of a possible control configuration that may beused to set the musical key used for a music-to-video game translation.The output sent to a video game system for a given set of musicalinformation differs for each key. The slider switches are used to setthe alphabetic key, whether the key is flat or sharp, and whether thekey is major or minor.

FIG. 5B is a diagram of a possible control configuration that may beused to set the musical key used for a music-to-video game translation.The output sent to a video game system for a given set of musicalinformation differs for each key. The arrow buttons are used to cyclethrough a set of preset or custom configurations.

FIG. 5C is a diagram of a possible control configuration that may beused to set the musical key used for a music-to-video game translation.The output sent to a video game system for a given set of musicalinformation differs for each key. The knobs are used to set thealphabetic key, whether the key is flat or sharp, and whether the key ismajor or minor.

DETAILED DESCRIPTION OF THE INVENTION

The invention is the use of a musical instrument as an input device fora video game. Rather than using a controller or a virtual guitar or acontroller simulating a musical instrument, a real musical instrument isused to control the action and play the video game.

Referring to FIG. 1, an electric guitar 100 is being played. The music110 being created is sent from the cable 120 connecting the guitar tothe device 130 that performs the translation for a video game system.The translated video game control information 140 is sent through acable connection from the device 130 to the video game system 150. Themusical 110 is also sent to an amplifier 160 which plays the actual,untranslated music aloud.

Referring to FIG. 2A, FIG. 2B, and FIG. 2C, there are other possiblepermutations for sending information from the guitar to the device thattranslates the music to video game input, then passing the informationon to the video game system. In FIG. 2A, this is a wireless connection200 from the guitar 201 to the translating device 202, then a cableconnection 203 from the translating device 202 to the video game system204. In FIG. 2B, this is a wireless connection 210 from the guitar 211to the translating device 212, then another wireless connection 213 fromthe translating device 212 to the video game system 214. In FIG. 2C,this is a cable connection 220 from the guitar 221 to the translatingdevice 222, then a wireless connection 223 from the translating device222 to the video game system 224.

Though the instrument described in conjunction with this game isfrequently the guitar, the instrument used to control the game can beany of a wide variety of musical instruments such as percussioninstruments (including cymbals, bell lyre, celeste, chimes, crotales,glockenspiel, marimba, orchestra bells, steel drums, timpani,vibraphone, xylophone, bass drum, crash cymbal, gong, suspended cymbal,tam-tam, tenor drum, tom-tom, acme siren, bird whistle, boat whistle,dinner bell, doorbell, finger cymbals, flex-a-tone, mouth organ,marching machine, police whistle, ratchet, rattle, sandpaper blocks,slapstick, sleigh bells, tambourine, temple blocks, thunder machine,train whistle, triangle, vibra-slap, wind machine, wood block, agogobells, bongo drum, cabaca, castanets, claves, conga, cowbell, maracas,scraper, timbales, kick drum, hi-hat, ride cymbal, sizzle cymbal, snaredrum, and splash cymbal), wind instruments (including piccolo, altoflute, bass flute, contra-alto flute, contrabass flute, subcontrabassflute, double contrabass flute, piccolo clarinet, sopranino clarinet,soprano clarinet, basset horn, alto clarinet, bass clarinet, contra-altoclarinet, contrabass clarinet, octocontra-alto clarinet, octocontrabassclarinet, kazoo, saxonette, soprillo, sopranino saxophone, sopranosaxophone, conn-o-sax, clar-o-sax, saxie, messo-soprano saxophone, basssaxophone, contrabass saxophone, subcontrabass saxophone, tubax,aulochrome, tarogato, folgerphone, contrabassoon, tenoroon, piccolooboe, oboe d'amore, English horn, French horn, oboe de caccia, bassoboe, baritone oboe, contrabass oboe, bagpipes, bugle, cornet,didgeridoo, euphonium, flugelhorn, shofar, sousaphone, trombone,trumpet, tuba, accordion, concertina, harmonica, harmonium, pipe organ,voice, bullroarer, lasso d'amore, whip, and siren), stringed instruments(including harps, electric bass, dulcimer, archlute, arpeggione, banjo,cello, Chapman stick, cittern, clavichord, double bass, fiddle, slideguitar, steel guitar, harpsichord, hurdy gurdy, Jew's harp, kora, koto,lute, lyre, mandola, mandolin, sitar, ukulele, viola, violin, andzither), keyboard instruments (including accordion, bandoneon, calliope,carillon, celesta, clavichord, glasschord, harpsichord, electronicorgan, Hammond organ, pipe organ, MIDI keyboard, baby grand piano,electric piano, grand piano, janko piano, toy piano, upright piano,viola organista, and spinets), and other instruments (turntable,computer-generated tones, frequency generator, klaxon).

The information used to register the musical information can bedetermined from the sound made by the instrument, or from the output ofthe musical instrument (such as the cord jack from a guitar), or from acomponent or components of the musical instrument (such as guitarpickups).

Referring to FIG. 3A, the pickups 300 on an electric guitar areindividually connected to a device 310 that translates the musicalinformation 320 from each, individual guitar pickup to video game input330 that is sent to a video game system 335.

Referring to FIG. 3B, an acoustic guitar 340 is being played in front ofa microphone 350 that is picking up the musical information 360 andsending it to a device 370 that translates the musical information 360to video game input 380 that is sent to a video game system 385.

After the musical information has been registered, the information fromthe instrument or instruments is converted to input information that canbe understood by the video game or video game system.

The musical information associated with a given input to the video gamecan be changed by configuration (such as to match the notes or chordsassociated with a certain musical key). The information can be a singlenote, a single beat, a set of notes played together (such as in achord), set of beats played together, a set of notes played in sequence,a set of beats played in sequence, or any combination thereof. Theconfiguration associating certain notes or sets of notes with inputinformation sent to the video game or video game system can beconfigured by key, by intended song, by the game for which it isintended, by the system for which it is intended, by the band for whichit is intended, a custom configuration, or any combination of theaforementioned methods of configuration.

Referring to FIG. 5A, FIG. 5B, and FIG. 5C, the device for translatingmusical information to video game input is based upon the desired key.In FIG. 5A, the key used for translating information is set by movingthe slider switches 501, 502, and 503; in this example, it is set toA-major. In FIG. 5B, the key used for translating information is set bycycling through a list of options using the up and down arrow buttons511 and 512 and viewing the selected configuration in the display 513;in this example, it is set to A-major. In FIG. 5C, the key used fortranslating information is set by turning the knob switches 521, 522,and 523; in this example, it is set to A-major.

The translation configurations may also be set up through software. Thiscan include providing users with the ability to build their own customconfigurations.

To provide an example, if a person is playing the guitar on an Xbox 360and the musical instrument translator is set to the key of E, the E notemay indicate to the Xbox 360 that the green A button was pressed, the Anote may be the blue X button, and the B note may be the yellow Ybutton. Furthermore, the E chord may be the red B button, the A buttonmay be the blue X button and the red B button, and the B chord may bethe green A button and the yellow Y button.

However, the same translator may then have its configuration changed tothe key of D. In that situation, the D note may indicate to the Xbox 360that the green A button and the yellow Y button were pressed, the G notemay be the blue X button, and the A note may be the yellow Y buttonalone. Furthermore, the D chord may be the green A button, the A chordmay be the blue X button and the red B button, and the B chord may bethe green A button and the yellow Y button.

Furthermore, the changes to the musical information that occur due tosuch techniques as bending strings, using the whammy bar (also known asthe vibrato bar), hammer-ons, pull-offs, holding notes, wah-wah pedals,reverbs, tremolos, pick slides, screeches, and turning the tuning keycan also be translated for use by the video game or video game system.The musical techniques can also include other special techniques forinstruments such as using mutes or muffles, using pitch bends wheels,plucking strings, and any other special technique that is performed by amusician or any other device that is used while playing an instrument.

The information translated can be a specific note in a specific octave,a specific note in any octave, a specific chord shape, any named chordof any shape, a specific note played in a certain way (such as a noteplayed on a certain string), a note played in any manner, and anycombination of the aforementioned information.

The information translated can be the creation of a sound, such as thestart of a note or the beat of a drum. The information translated can bechanged for the initiating event or percussion instrument used.

Referring to FIG. 4A, a snare drum 400 sensor 405, and an electronicdrum 410 are connected to a device 420 that translates information aboutbeats to video game input 430. When the drumsticks 440 and 450 strikethe drums 400 and 410, the beats are sent from the sensor 405 and theelectronic drum 410 to the device 430, which translates the informationabout the beats to the video game input 430 that is sent to the videogame system 460.

Referring to FIG. 4B, sensors 470 are attached to several differentdrums so that a theoretically unlimited number of inputs can be used bythe one device 480 used to translate musical information to video gameinput that can be sent along to a video game system 490.

The musical information from the instrument may also be sent to anamplifier so that the person playing will have the ability to hear themusic created by the instrument outside of its representation in a videogame or on a video game system.

Refer to FIG. 1, FIG. 2A, FIG. 2B, and FIG. 2C for graphical examples ofinformation being sent both to video game systems and amplifiers formusical instruments.

Though the expectation is that the musical instrument is generallyplayed with music simulators, the input may also be used for other typesof video games including but not limited to first-person shooter,survival horror, action adventure, fighting games, role playing games,real-time strategy games, platformers, puzzle games, sports games,rhythm action games, racing games, stealth-action games and third-personshooters.

Having described certain embodiments of the invention, other embodimentsincorporating the concepts of the invention may also be used. Althoughthe described concepts fall generally into the field of music simulatorswithin the genre of the rhythm action game, the principles of theinvention can extend to other areas of video game creation, development,concepts, ideas, usages, utilizations, competition, collaboration, andfunction. Therefore, the invention should not be limited to certainembodiments, but rather should be limited only by the spirit and scopeof the following claims.

1. The use of a musical instrument as an input device for a video gameor video game system.
 2. The use of a component or components of amusical instrument as an input device for a video game or video gamesystem.
 3. The translation of music or musical information into a formof input that can be recognized by a video game or video game system. 4.The use of claim 3 wherein a musical instrument communicates with amusic-to-input translator through a cable connection, through a plugconnection, through a wireless connection, through a device attached tothe instrument, through a device inside the instrument, through amicrophone picking up the sound of the instrument, or through a computerprogram.
 5. The use of claim 3 wherein a music-to-input translatorcommunicates with a video game or video game system through a cableconnection, through a plug connection, through a wireless connection,through a device attached to the video game system, through a deviceinside the video game system, or through a computer program.
 6. Themethod of claim 3 wherein the frequency, period, or amplitude of aparticular tone, note, or beat is translated into an input or a set ofinputs sent to a video game or video game system.
 7. The method of claim3 wherein the event of the creation of a tone, note, or beat istranslated into an input or a set of inputs sent to a video game orvideo game system.
 8. The method of claim 3 wherein a set of notes,tones, or beats played concurrently is translated into an input or a setof inputs sent to a video game or video game system.
 9. The method ofclaim 3 wherein a set of notes, tones, or beats played sequentially istranslated into an input or a set of inputs sent to a video game orvideo game system.
 10. The method of claim 3 wherein a change in thefrequency, period, or amplitude of a note, tone, or beat is translatedinto an input or set of inputs sent to a video game or video gamesystem.
 11. The method of claim 3 wherein the originating source of thenote, tone, or beat, or a set of notes, tones, or beats is translatedinto an input or a set of inputs sent to a video game or video gamesystem.
 12. The use of claim 3 wherein a note, tone, beat or a set ofnotes, tones, or beats can be translated into a different input or setof inputs sent to the video game or video game system, depending uponthe setting or configuration of the translator.
 13. The method of claim12 wherein differing notes, tones, or beats can be translated into oneinput or one set of inputs sent to the video game or video game system.14. The method of claim 12 wherein one note, tone or beat or one set ofnotes, tones, or beats is translated into different input depending uponthe relationship in time to other notes, tones, or beats.
 15. The methodof claim 12 wherein one note or tone or one set of notes or tones istranslated into different input depending upon the relationship in pitchto other notes or tones.
 16. The method of claim 12 wherein one note,tone, or beat or one set of notes, tones, or beats is translated intodifferent inputs depending upon the originating instrument.
 17. Themethod of claim 12 wherein the change to one note, tone, or beat or oneset of notes, tones, or beats is translated into different inputdepending upon the way in which the note, tone, or beat is changed.